test: exercise every Cloud read command against a live stack - #60
Open
JacobPEvans-personal wants to merge 1 commit into
Open
test: exercise every Cloud read command against a live stack#60JacobPEvans-personal wants to merge 1 commit into
JacobPEvans-personal wants to merge 1 commit into
Conversation
Cloud had three hand-written ACS checks while Enterprise had a catalog-driven suite covering every read leaf. The gap mattered most where Cloud is weakest: coverage is read-only and partial, so the behavior worth proving is that an unsupported command fails cleanly instead of falling through to an endpoint the stack does not serve. The new suite runs the same catalog the Enterprise suite uses. Reads that Cloud serves must return a typed data envelope; every other read must return a typed error envelope and a documented exit code. The must-succeed set is derived from the real dispatch table, so adding a Cloud route extends the suite instead of letting it drift. The credential gate fails loudly on a missing token or a non-Cloud URL, so a misconfigured canary cannot pass by skipping every test. Assisted-by: Claude:claude-fable-5 Claude-Session: https://claude.ai/code/session_0117oTjSHHjg1nh28LGRZ7My
JacobPEvans-personal
force-pushed
the
test/cloud-read-catalog
branch
from
August 1, 2026 01:08
26644ea to
caf5bc2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #59.
What
tests/integration/cloud/, mirroringtests/integration/enterprise/:conftest.py— one credential gate for the whole Cloud tree.read/test_catalog.py— every catalogued read, run against a live stack.read/test_acs_operations.py— the previoustest_acs_live.py, moved.Why
Enterprise had catalog-driven coverage of all 61 reads. Cloud had three hand-written ACS checks. The gap mattered most where Cloud is weakest: coverage is read-only and partial, so the guarantee worth proving is that an unsupported command fails cleanly rather than guessing at an endpoint the stack does not serve.
The contract it asserts
index list,role list,hec-token list) and offlineinspect{data, meta}envelope{error}envelope — never a traceback, never a silent fallthroughThe must-succeed set is derived from the real dispatch table (
has_cloud_list), not hardcoded, so adding a Cloud route extends this suite automatically instead of letting the test drift from the code.Proof
Note
pyproject.tomlswitches to--import-mode=importlibwithpythonpath = ["tests"]so the Enterprise and Cloud suites can share thetest_catalog.pyfilename. Without it pytest cannot distinguish two same-named test modules.